Watermark embedding apparatus and method through image structure conversion

ABSTRACT

One aspect of the present invention discloses a watermark embedding method. The method includes: inputting an original bitstream; determining a first frame for embedding a watermark in the input original bitstream; selecting one of frames after the first frame as a second frame; generating the first frame a bidirectional-coded frame (B frame) referring to the second frame; generating the second frame as a reference frame of the first frame; and embedding the watermark in the first frame generated as the B frame.

This application claims the benefit of priority of Korean PatentApplication No. 10-2017-0019530 filed on Feb. 13, 2017, which isincorporated by reference in its entirety herein.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to a watermark embedding apparatus and awatermark embedding method, and more particularly, to a watermarkembedding apparatus and a watermark embedding method for efficientlyenhancing robustness of watermark insertion.

Discussion of the Related Art

Digital watermarking is a technique used for protecting copyrights anddetermining forgery of digital contents by embedding non-cognitiveunique information in various literary productions including an audio,an image, a moving picture, and the like and enabling tracking theliterary productions through extraction of the embedded informationafterwards. In particular, a trend that illegal outflow and distributionof moving picture contents such as a movie or a TV broadcast rapidlyincrease is shown, and as a result, as a countermeasure against, awatermark technique is emerging.

FIG. 1A is a basic conceptual view of watermark embedding.

Referring to FIG. 1A, the watermark embedding apparatus generates awatermarked image I′ by embedding a watermark pattern in an image Iinput within a range in which an original is audiovisually similarlyviewed by configuring a watermark pattern W. In this case, watermarkinformation is modulated by using a secret/public key K to generate awatermark pattern having the same size as an original image. The secretkey is used for a purpose of preventing the watermark information frombeing interpreted by an authorized user and preventing the watermark andanother watermark from interfering with each other by allowing thewatermark patterns of both watermarks to be orthogonal to each otherwhen another watermark is inserted into contents in which the watermarkis already embedded by using the same watermark technique.

FIG. 1B is a basic conceptual view of watermark detection.

Referring to FIG. 1B, a watermark detecting apparatus may detect awatermark through the secret/public key K by receiving a test image I′.In this case, it is determined whether the detected watermark is thesame as the watermark W originally embedded by the embedding apparatusto determine whether to attack the corresponding image.

The contents in which the watermark is embedded are intentionally orunintentionally attacked under various environments and need to haverobustness to survive even after being attacked. In the case of themoving picture contents, the moving picture contents are subjected to acompression operation several times until the moving picture contentsare transferred from a content provider to the user and each step isperformed through lossy compression, and as a result, an embeddedwatermark signal cannot be avoided. Moreover, in recent years, with thedevelopment of compression technology, as compression efficiency hasfurther increased, a risk of the watermark signal embedded in alow-frequency band has been further increased in a compression step inorder to secure invisibility.

The compression is naturally performed in a transmission step, while acompression attack having a malicious purpose is even stronger incompression strength, and as a result, a countermeasure associated withmoving picture compression for securing the robustness of the watermarkis required.

As moving picture encoding, a method is primarily used, which decides akey frame through conditions including a scenecut among severalconsecutive frames, and the like and thereafter, encodes only adifference value from similar frames. In this case, the key frame issubjected to lossless compression and since other frames which refer tothe key frame may not have a lot of information, compression of otherframes becomes the lossy compression. When the watermark signal is to beembedded in the frame in which the lossy compression is performed,almost most signals disappear, and as a result, it is difficult todetect the watermark.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a watermark embeddingmethod and a watermark embedding apparatus which performs a structuralchange of a bitstream before embedding a watermark in order to enhancerobustness of the watermark.

In accordance with an embodiment of the present invention, a watermarkembedding method may include: inputting an original bitstream;determining a first frame for embedding a watermark in the inputoriginal bitstream; selecting one of frames after the first frame as asecond frame; generating the first frame a bidirectional-coded frame (Bframe) referring to the second frame; generating the second frame as areference frame of the first frame; and embedding the watermark in thefirst frame generated as the B frame.

The generating of the second frame as a reference frame of the firstframe may comprises generating the second frame as an intra-coded frame(I frame).

The first frame may be a scenecut frame or the I frame in the originalbitstream.

In a GoP structure changed by changing a reference relationship of thefirst frame and the second frame, the first frame generated as the Bframe may refer to the second frame in another GoP by activating afunction to refer to another GoP.

When the first frame is generated as the B frame, the GoP structure maybe changed so that the first frame generated as the B frame belongs to afirst GoP which is the previous GoP of a second group of picture (GoP)to which the first frame belongs in the original bitstream.

The GoP structure may be changed by changing and setting the first GoPto the extent of a frame immediately before the second frame.

The second frame may be a frame immediately after the first frame.

The determining of the first frame for embedding the watermark in theinput original bitstream may include selecting input image frames in theoriginal bitstream by the unit of n frames while shifting the inputimage frames at the interval of h frames, extracting a candidate frameof the selected n frames by using at least one of a plurality of codecsand scenecut detection algorithms, detecting a frame having a countvalue larger than a threshold value as a final scenecut frame throughhistogram analysis based on the detected candidate frame, anddetermining the detected final scenecut frame as the first frame, and hand n may be real numbers larger than 0 and h may have a smaller valuethan n.

In accordance with another embodiment of the present invention, awatermark embedding apparatus may include: an input unit inputting anoriginal bitstream; an embedding frame determining unit determining afirst frame for embedding a watermark in the input original bitstream; areference frame selecting unit selecting one of frames after the firstframe as a second frame; a reference frame generating unit generatingthe second frame as a reference frame of the first frame; a framechanging unit generating the first frame as a bidirectional-coded frame(B frame) referring to the second frame; and a watermark embedding unitembedding the watermark in the first frame generated as the B frame.

The reference frame generating unit may generate the second frame as anintra-coded frame (I frame).

The first frame may be a scenecut frame or the I frame in the originalbitstream.

In a GoP structure changed by changing a reference relationship of thefirst frame and the second frame, the first frame generated as the Bframe may refer to the second frame in another GoP by activating afunction to refer to another GoP.

When the first frame is generated as the B frame, the GoP structure maybe changed so that the first frame generated as the B frame belongs to afirst GoP which is the previous GoP of a second group of picture (GoP)to which the first frame belongs in the original bitstream.

The GoP structure may be changed by changing and setting the first GoPto the extent of a frame immediately before the second frame.

The second frame may be a frame immediately after the first frame.

The embedding frame determining unit may include an image selecting unitselecting an input image frames in the original bitstream by the unit ofn frames while shifting the input image frames at the interval of hframes, a candidate frame extracting unit extracting a candidate frameof the selected n frames by using at least one of a plurality of codecsand scenecut detection algorithms, and a final frame detecting unitdetecting a frame having a count value larger than a threshold value asa final scenecut frame through histogram analysis based on the detectedcandidate frame, the detected final scenecut frame is determined as thefirst frame, and h and n may be real numbers larger than 0 and h mayhave a smaller value than n.

By a watermark embedding method and a watermark embedding apparatusaccording to one aspect of the present invention, since an embeddingposition of a watermark can be determined for the first time, anincrease in capacity is minimized and detection rate is enhanced.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a schematic view of watermark embedding;

FIG. 1B is a schematic view of watermark detection;

FIG. 2 is a schematic view for describing a method for embedding awatermark in an I frame;

FIG. 3 is a schematic view for describing for a method for changing awatermark embedding target frame into a B frame by a watermark embeddingmethod according to an embodiment of the present invention;

FIG. 4 is a schematic view for describing for the method for changingthe watermark embedding target frame into the B frame when there is alimit in the number of GOPs in the watermark embedding method accordingto the embodiment of the present invention;

FIG. 5 is a block diagram schematically illustrating a watermarkembedding apparatus according to an embodiment of the present invention;

FIG. 6 is a detailed block diagram illustrating, in detail, a reencodingunit of the watermark embedding apparatus according to the embodiment ofthe present invention;

FIG. 7 is a detailed block diagram illustrating, in detail, an embeddingframe determining unit of the watermark embedding apparatus according tothe embodiment of the present invention;

FIG. 8 is a schematic view for describing a process of determining anembedding target frame by selecting a predetermined unit of image frame;

FIG. 9 is a diagram illustrating a histogram analysis result;

FIG. 10 is a table for comparing a change in capacity through typeconversion of an I frame and the subsequent frame by a watermarkembedding method according to an embodiment of the present invention;and

FIG. 11 is a table for comparing a capacity variation of an image inwhich a watermark is embedded and an original image by the watermarkembedding method according to the embodiment of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The present invention may have various modifications and variousembodiments and specific embodiments will be illustrated in the drawingsand described in detail.

However, this does not limit the present invention to specificembodiments, and it should be understood that the present inventioncovers all the modifications, equivalents and replacements includedwithin the idea and technical scope of the present invention.

Terms such as first, second, and the like may be used to describevarious components and the components should not be limited by theterms. The terms are used only to discriminate one constituent elementfrom another component. For example, a first component may be referredto as a second component, and similarly, the second component may bereferred to as the first component without departing from the scope ofthe present invention. A term ‘and/or’ includes a combination of aplurality of associated disclosed items or any item of the plurality ofassociated disclosed items.

It should be understood that, when it is described that a component is“connected to” or “accesses” another component, the component may bedirectly connected to or access the other component or a third componentmay be present therebetween. In contrast, it should be understood that,when it is described that an element is “directly connected to” or“directly access” another element, it is understood that no element ispresent between the element and another element.

Terms used in the present application are used only to describe specificembodiments, and are not intended to limit the present invention. Asingular form may include a plural form if there is no clearly oppositemeaning in the context. In the present application, it should beunderstood that term “include” or “have” indicates that a feature, anumber, a step, an operation, a component, a part or the combinationthereof described in the specification is present, but does not excludea possibility of presence or addition of one or more other features,numbers, steps, operations, components, parts or combinations thereof,in advance.

If it is not contrarily defined, all terms used herein includingtechnological or scientific terms have the same meanings as thosegenerally understood by a person with ordinary skill in the art. Termswhich are defined in a generally used dictionary should be interpretedto have the same meaning as the meaning in the context of the relatedart, and are not interpreted as an ideal meaning or excessively formalmeanings unless clearly defined in the present application.

Hereinafter, preferred embodiments of the present invention will bedescribed in detail with reference to the accompanying drawings. Indescribing the present invention, like reference numerals refer to likeelements in the drawings for easy overall understanding and a duplicateddescription of like elements will be omitted.

Throughout the present specification, a group of picture (GoP) as abasic unit of coding/edition include at least one I frame and may begenerally constituted by 10 to 15 frames. It is preferable that the GOPstarts with the I frame or a B frame and ends with the I or B frame andwhen the GoP starts with the B frame, the GoP may be reconstructed byreferring to the subsequent I frame.

The intra-coded frame means a frame decoded without referring to anotherpicture by intra prediction and P and B frames mean frames decoded byreferring other pictures, which include the I frame. Herein, thepredictive-coded frame (P frame) means a frame that refers aunidirectional frame and the bidirectional-coded frame (B frame) means aframe that refers a bidirectional frame.

The picture and the frame as terms designating individual images may beused mixedly with each other.

Further, a watermark embedding apparatus according to an embodiment ofthe present invention may be implemented by user terminals such as acontent provider server providing contents through a network, a settopbox, and a PC, and the like.

Embedding Watermark in I Frame

FIG. 2 is a schematic view for describing a method for embedding awatermark in an I frame.

Referring to FIG. 2, there are a lot of cases in that in an image, ascenecut frame and/or a key frame of the image are the I frames.Therefore, the I frame may be more robust to an attack or fake of theimage than other frames. The reason is that since the scenecut frame issubstantially important information, the scenecut frame is generallydesignated as the I frame in the image.

In the embodiment of FIG. 2, a situation in which the watermark isembedded in an I frame 222 which is almost lossless is assumed. In thiscase, GoP 1 210 ends with the B frame and thereafter, there is a highpossibility that the first frame of GoP 2 220 will be the scenecutframe. Accordingly, the I frame 222 is general.

In this case, when the watermark is embedded in the I frame 22 whichwill be the scene switching frame with a high probability, a probabilitythat the watermark will survive is high, but reencoding is required andthe B and P frames in the GoP 2 220 generally directly/indirectly referto the I frame 222, and as a result, the number of frames transformedalso increases with transformation of the I frame 222, therebyincreasing encoding complexity. Further, since a lot of bit rate isallocated in the order of the I, P, and B frames in image compression,when the watermark is embedded in the I frame 222, there is a highpossibility that significant damage will be made in terms of thecapacity and the bit rate of the image.

Embedding of Watermark with Conversion of Frame Type

FIG. 3 is a schematic view for describing for a method for changing awatermark embedding target frame into a B frame by a watermark embeddingmethod according to an embodiment of the present invention.

Referring to FIG. 3, when an original bitstream is embedded in thewatermark embedding apparatus, the watermark embedding apparatusdetermines a target frame for embedding the watermark and recompressesthe determined target frame. In this case, the watermark embeddingtarget frame is preferably the scenecut frame and/or the I frame in theGoP. However, the watermark embedding target frame is not particularlylimited thereto and the P frame is also selectable.

As described above, when the scenecut frame is a variable GoP, there isa high possibility that the scenecut frame will be the I frame and whenthe scenecut frame is a fixed GoP, there is a high possibility that thescenecut frame will be the I or P frame.

In the embodiment of the present invention, the watermark embeddingapparatus may select an I frame 322 which is the first frame of a GoP 2320 as the target frame for embedding the watermark. Since detection ofthe scenecut frame or the I frame occupies an important part inwatermark embedding efficiency of the present invention, a method formore accurately detecting the scenecut frame or the I frame is requiredand the method will be described below in more detail with reference toFIGS. 7 to 9.

Moreover, in the embodiment of the present invention, the watermarkembedding apparatus preferably embeds a watermark robust to adeterioration attack in each frame by generating the watermark with aframe based watermark. To this end, it is preferable to prevent awatermark corresponding to noise from being lost and compressed whileencoding by analyzing a feature of the frame. Accordingly, it ispreferable to embed the watermark in a part corresponding to a lowfrequency in a discrete cosine transform (DCT) filtering process whichis one process of the encoding. Alternatively, the robustness may beincreased by embedding the robust watermark in an edge of the image oran area not well viewed by a person.

When the watermark embedding apparatus selects the I frame 322 as thetarget frame in which the watermark will be embedded, the watermarkembedding apparatus forcibly encodes the I frame 322 into a B frame 332in order to prevent the capacity of the image from being increased andincrease bit rate efficiency.

If the watermark embedding target frame is the P frame, since multiplesimilar frames exist in the same GoP, a next frame may be just changedto the p frame and a separate additional work is not required. However,as described in the embodiment of FIG. 3, when the embedding targetframe is the I frame, the GoP is double-speed up forward, and as aresult, encoding efficiency significantly deteriorates. The reason isthat since there is no similar frame in the same GoP, an area to bereferred does not exist, and as a result, all most areas may bedetermined as an intra-coded area. Therefore, while the I frame isforcibly changed to the B frame 332 and the B frame 332 is enabled torefer to another GoP, one frame 324 among frames disposed later ischanged to the I frame 334 and thereafter, the corresponding frame 334is allowed to be referred, thereby increasing encoding efficiency.

When this is described in detail, in the original bitstream, apredetermined frame 324 after the I frame 322 which is the embeddingtarget frame is selected to encode the selected frame into the I frame334. In this case, the frame 324 selected for type conversion of theframe in the original bitstream is not limited to the P frame and the Bframe is also available and the frame 324 need not be particularlychanged to the I frame. The P frame including a lot of intra predictionmay also be used.

When such frame type conversion is achieved, the B frame 332 into whichthe I frame 322 is frame-type converted belongs to the GoP 1 310 whichis the previous GoP of the GoP 2 320 to which the B frame 332 belongs inthe original bitstream and while the P frame 324 selected for typeconversion into the I frame is encoded into the I frame 334, the P frame324 becomes the first frame in the GoP 2 320 which is the subsequent GoPof the GoP 1 to which the previous B frame 332 belongs. Further, the GoP1 310 may include even the previous frame (B frame) of the I frame 334.

In this case, since the I frame 334 into which the P frame 323 isframe-type converted is converted to serve as a reference frame of the Bframe 332, the B frame 332 in the watermarked image refers to the Iframe 334 of not the GoP to which the B frame 332 belongs but the GoP 2320. In order to smoothly induce this case, it is preferable to use anopen GoP option which may have a reference relationship over a GoP unit.Further, the B frames in the GoP 2 320 which refers to the I frame 322in the original stream is also encoded by resetting the referencerelationship so as to refer to the I frame 334 reencoded while the Pframe 334 is frame-type converted in the watermarked image.

FIG. 4 is a schematic view for describing for the method for changingthe watermark embedding target frame into the B frame when there is alimit in the number of GOPs in the watermark embedding method accordingto the embodiment of the present invention.

Referring to FIG. 4, the watermark embedding apparatus forcibly sets aframe in which the watermark embedding apparatus wants to embed thewatermark as the B frame and sets the B frame as an end frame of theGoP. In some cases, protocols such as HTTP live streaming (HLS) usedover the top (OTT) and dynamic adaptive streaming over HTTP (DASH) orthe GoP in a live broadcast may be forcibly determined. In theembodiment of FIG. 4, it is assumed that the GoP is fixed to 12.

In the case of the OTT, in the original bitstream, the I frame may bearbitrarily encoded into the B frame and the immediately next frame maybe encoded into the I frame. Since the frame which is shown immediatelyafter the I frame is a frame which is most similar to the I frame, whenthe frame is forcibly encoded into the I frame, the encoded I frame maynot significantly influence overall image encoding. In addition, whenthe immediately next frame is changed to the I frame, only the B frame(expressed as Bw in FIG. 4) in which the watermark is embedded isincluded in the previous GoP and all GoPs may continuously maintain thesize of the GoP while only one frame is pushed. In the case of the livebroadcast, one GoP is wholly encoded and only one frame is referred inthe next GoP and thereafter, when the size of the corresponding GoP isdetermined, it is possible to sufficiently embed the watermark.

Moreover, such a filtering operation is configured so as to preventanother frame from referring to information on the B frame because theinformation on the B frame is broken by embedding the watermark in the Bframe and controlling B-pyramid and OpenGoP options.

According to another embodiment of the present invention, the watermarkmay be embedded in an environment in which the frame is forcibly changedby configuring frame types of 12 respective frames constituting the GoPto be arranged in a specific frame type order. Further, a filteringoperation may be performed, which embeds and encodes the watermark by aquantization parameter (QP) value of the I frame in which the watermarkis embedded so as to allow the watermark to survive in variousdeterioration codomains.

Configuration of Watermark Embedding Apparatus

FIG. 5 is a block diagram schematically illustrating a watermarkembedding apparatus according to an embodiment of the present invention.Referring to FIG. 5, the watermark embedding apparatus may include aninput unit 510, a decoding unit 520, and a re-encoding unit 530.Respective constituent elements may be implemented by one hardwareprocessor and a plurality of constituent elements may be integrallyimplemented by one hardware processor. Further, commands or data relatedwith the respective constituent elements are stored in a memory (notillustrated) connected with a processor and the related data may beprovided to the processor.

Referring to FIG. 5, the input unit 510 receives an original bit stream.

The decoding unit 520 decodes the original bit stream input through theinput unit 510 to acquire information on each frame.

The re-encoding unit 530 determines a watermark embedding target framebased on the decoded frame to convert a frame type of the correspondingframe into a B frame, and then embeds the watermark. In addition, one ofsubsequent frames of the embedding target frame is selected and theselected frame is converted into a I or P frame and then encoded bynewly setting a reference relation according to a GoP structure changedaccording to the converted frame. The more detailed contents for theconfiguration of the re-encoding unit 530 will be described withreference to FIG. 6.

In the embodiment of the present invention, except for the decoding unit520, while the decoding for all frames is not performed, a type changeof the frame and the watermark embedding process may be performed by there-encoding unit 530.

FIG. 6 is a detailed block diagram illustrating, in detail, there-encoding unit of the watermark embedding apparatus according to theembodiment of the present invention. As illustrated in FIG. 6, there-encoding unit according to the embodiment of the present inventionmay include an embedding frame determining unit 610, a reference frameselecting unit 620, a B frame generating unit 630, a reference framegenerating unit 640, a reference relation changing unit 650, and awatermark embedding unit 660.

Referring to FIG. 6, the embedding frame determining unit 610 determinesa target frame for embedding the watermark in the original bit streaminput through the input unit (not illustrated). The embedding targetframe may select a scenecut frame.

The reference frame selecting unit 620 selects one of the subsequentframes of the embedding target frame determined in the embedding framedetermining unit 610 as a target to be converted into an I frame to bereferenced by the embedding target frame. The reference frame selectingunit 620 may select a frame immediately next to the embedding targetframe as a reference frame so as to minimize a change in the referencerelation with high similarity to the embedding target frame.

The B frame generating unit 630 encodes arbitrarily the embedding targetframe determined in the embedding frame determining unit 610 to a Bframe. The B frame changes reference information to refer to the frameselected in the reference frame selecting unit 620. That is, referencetarget frame index information and reference block information referringfor each coding block and/or prediction block may be changed accordingto the changed GoP structure.

The reference frame generating unit 640 generates the frame selected inthe reference frame selecting unit 620 as an I frame and/or a pluralityof P frame including intra prediction.

The reference relation changing unit 650 changes the reference relationof the frames that refer to the embedding target frame which has been anI frame before changing the frame type to change the reference relationof each frame to refer to the frame (the frame of which the frame typeis changed to the I frame) selected in the reference frame selectingunit 620.

The watermark embedding unit 660 embeds the watermark in the B framegenerated in the B frame generating unit 630.

Method of Detecting Accurate Scenecut Frame

FIG. 7 is a detailed block diagram illustrating, in detail, theembedding frame determining unit of the watermark embedding apparatusaccording to the embodiment of the present invention. As described inFIG. 7, the embedding frame determining unit according to the embodimentof the present invention may include an image selecting unit 710, acandidate frame detecting unit 720, and a final frame detecting unit730.

Referring to FIG. 7, the watermark embedding apparatus receives a movingpicture bit stream to detect a scenecut frame in the bit stream and mayselect the scenecut frame as a target frame for embedding the watermark.As described above, the scenecut frame and the I frame as the key frameof the image are strong against the attack on the image and thus findingaccurately the frames is important in enhancing robustness of embeddingthe watermark. The watermark embedding apparatus intends to select sucha scenecut frame or the I frame as the key frame as a target framearbitrarily converted to the B frame.

The image selecting unit 710 receives the inputted moving picture streamto divide and select the inputted moving picture stream into specificunits. The image selecting unit 710 selects a unit frame having aspecific size while being shift at h frame intervals to transmit theselected unit frame to the candidate frame detecting unit 720 to detecta candidate scenecut frame with respect to the selected frame. In thiscase, the unit frame may be set as n frames. Herein, n and h are realnumbers larger than 0. Further, h may be a value smaller than n.

The divided unit image frame is provided to the candidate framedetecting unit 720. The candidate frame detecting unit 720 may include aplurality of frame extracting units 722 and a weight calculating unit724 connected thereto. Each frame extracting unit 722 receives the unitimage frames divided by the image selecting unit 710 and extracts acandidate scenecut frame in each unit image. The frame extracting unit722 may include a plurality of codecs (an encoding/decoding unit).Alternatively, the frame extracting unit 722 may include a plurality ofscenecut detection algorithm executing units. Alternatively, the frameextracting unit 722 may be a combination of the codec and scenecutdetection algorithm executing unit.

A scenecut frame candidate in the corresponding unit image is found bythe frame extracting unit 722. Here, the candidate frame or thecandidate scenecut frame is not finally detected as the scenecut frame,but refers to a frame having a high possibility to be adopted as a finalscenecut frame by weight calculation or other algorithm (for example,histogram analysis). A plurality of candidate scenecut frames may bedetected in the selected image. In addition, the attributes of thecandidate scenecut frames may also be different from each other. Forexample, one frame may be an I frame, and the other frame may be a frameof the P frame or the B frame of which a ratio of the intra region ishigher than a certain threshold value. Herein, the I frame means a framedecoded without referring to another frame through intra prediction, theP frame may be a frame referring to a unidirectional picture, and the Bframe may be a frame referring to a bidirectional picture. Further, whenthe frame extracting unit 722 is the scenecut detection algorithmexecuting unit rather than the codec, a frame detected through thealgorithm may be immediately the candidate scenecut frame.

The frame extracting unit 722 may extract the candidate scenecut framewith respect to the overlapped unit image portion when a part of theunit image is overlapped because the h value is smaller than n.

The frame extracting unit 722 may receive frame time information orframe index information from a predetermined memory (e.g., a framebuffer) for each input frame. Then, after the candidate scenecut frameis extracted through the codec or the scenecut detection algorithmdescribed above, a flag indicating whether the candidate scenecut frameis or not is given or time information or index of the frame selected asthe candidate scenecut frame may be displayed and identified on a timeline. Through the above method, identification information on thecandidate scenecut frame may be shared with other components.

The weight calculating unit 724 multiplies an appropriate weight valueby the candidate scenecut frame selected through the frame extractingunit 722. The weight value may be given to each frame calculating unit724, and the value of the weighted candidate scenecut frame is providedto the final frame detecting unit 730.

The final frame detecting unit 730 may perform a histogram analysisbased on the value of the weighted candidate switch frame to detect acandidate frame having a count value higher than the threshold value asa final scenecut frame. Herein, the histogram analysis may be anoperation for obtaining summation of the weighted values. In addition,the final frame detecting unit 730 provides a flag to identify thescenecut frame to the final scenecut frame or records the index value ofthe final scenecut frame to immediately recognize the identifiedscenecut frame even in other devices below.

FIG. 8 is a schematic view for describing a process of determining anembedding target frame by selecting a predetermined unit of image frame.

Referring to FIG. 8, an image selecting unit (not illustrated) dividesthe input bit stream into n units to provide the divided units to acandidate frame detecting unit (not illustrated). Herein, the imageselecting unit may include a configuration for shifting the image by hframes and a configuration for selecting only n frames from the frontpart of the shifted section.

In the embodiment of the present invention, the image selecting unit maydivide a bit stream from 0 frame to 1000 frames. Herein, if the decodingis performed at a speed of 50 fps, the above 1000 frames may have a timeinterval of 20 seconds, and the frame may have respective timeinformation. At this time, h frames may be selected earlier in time withrespect to 0 frame which is the first part of the bit stream. That is, nframes are divided and selected from the concept (0-h) frame to providethe divided and selected frame to the candidate frame detecting unit.

Thereafter, the image selecting unit passes the input frame by h framesand divides and selects the n frames again for the subsequent section toprovide the selected frame to the candidate frame detecting unit. Thisprocess is repeated until it reaches 1000 frames. Even in the endportion of the bit stream, n unit images are selected to exceed 1000frames through the portion passing through the interval h so that imageframes in the bit stream may be selected without omission.

According to the embodiment of the present invention, h is preferablysmaller than n. If the h value is larger than n, the frame is shiftedwhen the next image is selected by the h value exceeding the n value,and thus some frames of the bit stream may not be selected. If the h andn values are the same as each other, the duplicated portion is notpresent and thus a scenecut frame that is not extracted may be present.For example, when the frame is divided in the GOP unit, the first frameof the next unit image may be excluded from the detection target, andthus in spite of the scenecut frame, the frame is not detected. In orderto prevent the problem, the present invention may be designed so thatthe duplicated frame is present, that is, the value h is smaller thanthe value n, and the scenecut portion may be detected not depending onone codec or scenecut detection algorithm but depending on a pluralityof codecs or algorithms.

Accordingly, it may be preferable that the divided and selected unitimages may be overlapped with respect to at least one frame. The valuesh and n may be defined through a user setting. The values h and n may bearbitrarily changed through a user interface (not illustrated). Forexample, the value n may have values of 60, 120, 240, . . . , and thelike. The value h may have values of 10, 20, 7.5, . . . , and the like.In some cases, the value h may be set to a time unit, not a frame unit,and may be set to 0.2 second, 0.4 second, 0.15 second, and the like at50 fps. That is, the units of h and n are not necessarily the same aseach other.

According to the embodiment of the present invention, the value n may bea value representing a specific GOP unit. However, even in this case,the h value may have a different value from the GOP value.

As such, a detailed analysis in the unit frame with respect to thedivided image is performed. The h frame moves within the unit frame, andthen the image is divided by n frames in the whole frame to search thescenecut frame or the I frame. In this case, the detailed analysis isperformed through a plurality of codecs and/or scene transitiondetection algorithms. The histogram analysis is performed for each frameby multiplying the candidate scenecut frames selected through the codecand/or the scenecut detection algorithm by a weight. The histogramanalysis is performed by accumulating all the weight values calculatedfor each frame using a frame axis and calculating the count value. Thecount value is calculated for each individual frame and the upper 60% ofthe specific frame period by applying the threshold value based on thecalculated count value, and the corresponding frame may be detected asthe final scenecut frame. Alternatively, the threshold value is set as aspecific value of 50 to detect the frame having a counter value higherthan the threshold value as the final scenecut frame. In this case, theupper percentage ratio and the threshold value for detecting thescenecut frame may be defined through the user setting.

FIG. 9 is a diagram illustrating a histogram analysis result.

An example of FIG. 9 may be a history in which a weight calculationresult for 1000 frames from 0 to 1000 frames is extracted through thehistogram. Herein, if the threshold value is set to 20, a frame having acount value higher than 20 (indicated by ‘o’ in the drawing) may bedetected as a final scenecut frame. In this case, the threshold value isnot necessarily fixed to 20 as one count value, but can be arbitrarilychanged through the user interface. In addition, the specific countvalue is not set as the threshold value and a frame having an uppercount value within a specific interval such as the upper several % maybe detected as a final scenecut frame.

According to the embodiment of the present invention, a flag foridentifying the final scenecut frame detected as described above isallocated, or upper frame index information (alternatively, timeinformation) may be stored through a table or other data storage meansfor managing the frame. Through the index information, the watermarkembedding apparatus may identify the scenecut frame and/or the I frame,and may select the watermark embedding target frame based on theidentified frame. The selected frame may be forcibly encoded as the Bframe.

Simulation Result

FIG. 10 is a table comparing a change in capacity through typeconversion of an I frame and the subsequent frame by a watermarkembedding method according to an embodiment of the present invention.

Referring to FIG. 10, the capacity change between two frames changingthe frame type is substantially similar to the capacity before the typeof each frame is changed. In the simulation, it was verified that therobustness of the watermark from the external attack may be secured moreefficiently while minimizing the increase of the capacity of thewatermark-embedded frame through the method of forcibly converting the Iframe embedded with the watermark into the B frame.

FIG. 11 is a table for comparing a capacity variation of an image inwhich a watermark is embedded and an original image by the watermarkembedding method according to the embodiment of the present invention.

Referring to FIG. 11, in order to test the robustness of the watermarkembedding method according to an embodiment of the present invention, atest was performed on a sample image provided freely. The size of theoriginal image is 1920×800 and the bit rate is 6M. The file size of thisimage is 494 MB. The table in FIG. 11 illustrates the capacity due tothe frame variation. The capacity of the image was increased (about 9MB) enough to have no load on the system for transmitting the imagedata. Therefore, it can be verified that the robustness may be maximizedwithout imposing a burden on the size of the watermarked image in thecase of using the watermark embedding method according to the embodimentof the present invention.

The present invention has been described with reference to the preferredembodiments of the present invention. However, it will be appreciated bythose skilled in the art that various modifications and changes of thepresent invention can be made without departing from the spirit and thescope of the present invention which are defined in the appended claimsand their equivalents.

What is claimed is:
 1. A watermark embedding method comprising:inputting an original bitstream; determining a first frame for embeddinga watermark in the inputted original bitstream; selecting one of aplurality of frames as a second frame, wherein the second frame ispositioned after the first frame; converting a frame type of the firstframe from a non-bidirectional-coded frame to a bidirectional-codedframe (B frame), wherein the B frame is referring to the second frame;generating a reference relationship for the second frame as a referenceframe of the first frame; and embedding the watermark in the firstframe, which is converted to the B frame.
 2. The watermark embeddingmethod of claim 1, wherein the generating the reference relationship forthe second frame comprises converting a frame type of the second frameto an intra-coded frame (I frame).
 3. The watermark embedding method ofclaim 1, wherein the first frame is a scenecut frame or the I frame inthe original bitstream.
 4. The watermark embedding method of claim 1,wherein in a group of picture (GoP) structure changed by changing thereference relationship of the first frame and the second frame, thefirst frame, which is converted to the B frame refers to the secondframe in another GoP by activating a function to refer to another GoP.5. The watermark embedding method of claim 1, wherein, when the frametype of the first frame is converted to the B frame, the GoP structureis changed so that the first frame, which is converted to the B framebelongs to a first GoP which is the previous GoP of a second group ofpicture (GoP) to which the first frame belongs in the originalbitstream.
 6. The watermark embedding method of claim 5, wherein the GoPstructure is changed by changing and setting the first GoP to the extentof a frame immediately before the second frame.
 7. The watermarkembedding method of claim 1, wherein the second frame is positionedimmediately after the first frame.
 8. The watermark embedding method ofclaim 1, wherein the determining of the first frame for embedding thewatermark in the inputted original bitstream includes the steps of:selecting input image frames in the inputted original bitstream by aunit of n frames while shifting the input image frames with an intervalof h frames; extracting a candidate frame of the selected n frames byusing at least one of a plurality of codecs and scenecut detectionalgorithms; detecting a frame having a count value larger than athreshold value as a final scenecut frame through histogram analysisbased on the detected candidate frame; and determining the detectedfinal scenecut frame as the first frame, wherein h and n are realnumbers larger than 0, and h has a smaller value than n.
 9. A watermarkembedding apparatus comprising: an input unit for inputting an originalbitstream; an embedding frame determining unit for determining a firstframe for embedding a watermark in the inputted original bitstream; areference frame selecting unit for selecting one of a plurality offrames as a second frame, wherein the second frame is positioned afterthe first frame; a frame changing unit for converting a frame type ofthe first frame from a non-bidirectional-coded frame to abidirectional-coded frame (B frame), wherein the B frame is referring tothe second frame; a reference frame generating unit for generating areference relationship for the second frame as a reference frame of thefirst frame; and a watermark embedding unit for embedding the watermarkin the first frame, which is converted to the B frame.
 10. The watermarkembedding apparatus of claim 9, wherein the reference frame generatingunit further converts a frame type of the second frame to an intra-codedframe (I frame).
 11. The watermark embedding apparatus of claim 9,wherein the first frame is a scenecut frame or the I frame in theoriginal bitstream.
 12. The watermark embedding apparatus of claim 9,wherein in a group of picture (GoP) structure changed by changing thereference relationship of the first frame and the second frame, thefirst frame, which is converted to the B frame refers to the secondframe in another GoP by activating a function to refer to another GoP.13. The watermark embedding apparatus of claim 9, wherein, when theframe type of the first frame is converted to the B frame, the GoPstructure is changed so that the first frame, which is converted to theB frame belongs to a first GoP which is the previous GoP of a secondgroup of picture (GoP) to which the first frame belongs in the originalbitstream.
 14. The watermark embedding apparatus of claim 13, whereinthe GoP structure is changed by changing and setting the first GoP tothe extent of a frame immediately before the second frame.
 15. Thewatermark embedding apparatus of claim 9, wherein the second frame ispositioned immediately after the first frame.
 16. The watermarkembedding apparatus of claim 9, wherein the embedding frame determiningunit further includes; an image selecting unit for selecting input imageframes in the inputted original bitstream by a unit of n frames whileshifting the input image frames with an interval of h frames, acandidate frame extracting unit for extracting a candidate frame of theselected n frames by using at least one of a plurality of codecs andscenecut detection algorithms, and a final frame detecting unit fordetecting a frame having a count value larger than a threshold value asa final scenecut frame through histogram analysis based on the detectedcandidate frame, the detected final scenecut frame is determined as thefirst frame, wherein h and n are real numbers larger than 0, and h has asmaller value than n.